14. Project Submission and Getting Feedback

Project submission and review

To submit this project, you will be submitting either as a Team Lead or a Team Member. Only Team Lead submissions are tested - Team Member submissions are passed through our system automatically with no testing. This is so we have a submission record for each individual student, but do not have to test the same code multiple times for a multi-person team. Team Leads and Team Members have different submission instructions, so please help us out by following the correct set of instructions in the Project concept!

This project is unlike other projects this Nanodegree program so far, in that there is no formal code review or pass/fail status for this project. Once your project is submitted, it will be tested first in the Udacity simulator and then (if it passes the simulator testing) run on Carla, our self-driving car! Your Team Lead will receive the ROS bag data and log files from these tests to share with the team. You can use this data to observe your code’s performance, as well as to debug and modify your code. When your team is satisfied with the test results, your Team Lead will be able to close your submission and you will then be able to apply for graduation. We’ll go into how to do this in the Project concept.

Please note that we will not be able to accomodate special launch instructions or run additional scripts from your submission to download files - this includes any changes to the requirements.txt file.

Getting and using rosbag feedback

Once your team submits the project, the Team Lead’s submission will first be tested using a simulator to ensure that the vehicle is following waypoints and adhering to the km/h velocity limit set by the velocity rosparam in waypoint_loader . Once this is verified, our capstone reviewer will take your project to the Udacity test track for real-world testing on Carla!

After both the simulator test and the Carla test are complete, your Team Lead will receive a link to download the team’s feedback in the form of log files and a ROS bag recording of Carla on our test track. Note that it typically takes up to a week to receive feedback for this project, so plan accordingly.

The ROS bag file contains all messages passed on all topics in ROS while Carla is driving, providing your team with a full recording of all available data (including camera data) created at the time of testing. Note that the download link to this feedback expires after 20 days, so Team Leads should be sure to download all data and feedback promptly.

To replay the ROS bag using the workspace, use the following steps:

  1. Navigate to a temporary storage folder, so as not to exceed the 2GB limit in /home/workspace . Download the rosbag using wget and the download link: 
  cd /opt
  wget <link_to_your_download>
  1. Unzip the download: 
  unzip /opt/path/to/your/download.zip
  1. Open a terminal and start roscore .
  2. Open another terminal and run rosbag play -l /opt/path/to/your.bag
  3. Click the "Go To Desktop" button.
  4. From the XWindows desktop, open terminator, and run rviz . You can change the RViz configuration to Udacity's .config file by navigating to /home/workspace/default.rviz from File > Open Config in RViz.

If you are not using a workspace, use the following steps:

  1. Download the rviz config file here .
  2. Open a terminal and start roscore .
  3. Open another terminal and run rosbag play -l /path/to/your.bag
  4. Open one more terminal and run rviz . You can change the RViz configuration to the Udacity download by navigating to your config file from File > Open Config in RViz. Alternatively, if you'd like to make the Udacity config file your default, you can replace the rviz config file found in ~/.rviz/default.rviz .

You should see something similar to the following in RViz:

While the bag is playing, you can use ROS commands to list or echo topics, grep and pipe messages, or use any other debugging techniques that might be useful. This may be helpful, for example, if you wanted to see the ROS messages sent over the /vehicle/throttle_cmd topic as the vehicle was driving around the lot. If you've used logging in your ROS nodes, remember that all log messages except for debugging messages are published to the /rosout topic by default, so you will be able to view logging messages using the ROS bag as well. If you'd like debugging messages to also show up in the logs, be sure to set the log_level when initializing your ROS nodes: 

rospy.init_node('my_node', log_level=rospy.DEBUG)

Next Concept